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Abstract —  The  paper  explores  pre-service  teacher  ’sawareness,  interest,  perception,  attitudes  and  the  level  of 
knowledge  on  the  Programme  for  International  Student  Assessment  for  development  (PISA-D).  A  mixed  method  of 
quantitative  research  and  qualitative  research  was  used  in  this  study.  The  study  was  conducted  among  final  year 
Bachelor  of  Education  ( B.Ed )  and  Postgraduate  Diploma  in  Education  (PGDE)  students  at  Samtse  College  of 
Education,  Bhutan.  Instruments  such  as  survey  questionnaire  and  interview  were  used  to  collect  the  data.  The 
survey  questionnaire  was  analyzed  using  Excel  Add-in  Analysis  Tool  Pak  and  Interview  data  were  analyzed  by 
identifying  themes.  The  results  revealed  that  72.6%  of  pre-service  teachers  were  aware  of  Programme  for 
International  Student  Assessment  for  Development  (PISA  -D).  Among  them  nearly  40%  expressed  their  opinion  that 
PISA  has  impact  on  curriculum  and  quality  of  education  in  the  country  as  a  whole.  However,  60%had  limited 
knowledge  of  purpose  and  importance  of  participating  in  PISA-D.  The  findings  also  revealed  that  50%  of  them  were 
interested  to  know  more  about  PISA. 
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I.  BACKGROUND 


2017(Zangmo,  2017).  The  major  finding  indicated  that 
“students  performed  slightly  well  in  reading  and  scientific 
literacy  compared  to  mathematical  literacy.  It  found  that  poor 
reading  literacy  affected  students’  performance  in  science 
and  mathematics  because  they  [were]  unable  to  understand 
the  language”  (Rinzin,  2019).  The  finding  also  reported  that 
“girls  outdid  boys  in  reading  literacy  while  boys  performed 
better  than  girls  in  mathematical  literacy.  Both  boys  and  girls 
performed  almost  at  par  in  scientific  literacy”  (Rinzin). In  the 
PISA  reports,  teachers  are  described  as  actors  performing 
activities  that  are  important  for  reaching  the  goals  set  for 
education.  Teachers  are  also  hope  and  the  happenings  of 
educational  transformation  and  development 
(Perttersson&Molstad,  2016).  However,  the  PISA  is  neither 
taught  nor  discussed  in  the  educational  colleges  in  the 
country.  Surprisingly  teacher’s  opinion  on  PISA  has  not  yet 
been  studied,  although  teachers  play  very  important  role  in 


The  Programme  for  International  Student  Assessment  for 
Development  (PISA-D)  was  initiated  and  launched  by  an 
international  organisation  called  Organisation  for  Economic 
Cooperation  and  Development  (OECD)  and  its  partners  in 
2013  to  encourage  and  facilitate  PISA  participation  for 
interested  and  motivated  low  and  middle  income  countries 
(OECD, 2016).  The  main  aim  of  the  PISA-D  is  to  build  the 
capacity  to  manage  large-scale  student  learning  assessment 
and  to  support  policy  dialogue  and  decision  making  in  the 
participating  countries  likeCambodia,  Ecuador,  Guatemala, 
Honduras,  Panama,  Paraguay,  Senegal  and  Zambia  (OECD, 
2016).  Bhutan  joined  PISA-D  in  2017  to  see  how  Bhutanese 
students  fair  with  the  students  of  OECD  member  countries 
(The  Bhutanese,  2017). 


More  than  2000  Bhutanese  students  between  the  ages  of  15 
and  16  had  attended  the  PISA-D  test  in  November, 
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education  process  in  the  country.  Therefore  the  main 
objective  of  the  study  is  to  explore  pre-service  teachers’ 
awareness,  interest,  attitude  and  knowledge  on  PISA-D. 

II.  RESEARCH  QUESTION  AND  SUB¬ 
QUESTION 

The  following  research  questions  were  formulated  to  guide 
the  researcher  in  the  study. 

Main  question: 

What  are  the  opinions  (awareness,  perception,  attitudes, 
interest  and  knowledge)  of  pre-service  teachers  towards 
PISA-D  test? 

Sub  questions 

1.  Are  pre-service  teachers’  aware  of  PISA-D  test? 

2.  What  are  pre-service  teachers’  perceptions  and 
attitudes  towards  PISA-D  test? 

3.  To  what  extent  pre-service  teachers  are  interested  in 
PISA-D  test  result  of  Bhutanese  school  students? 

4.  What  is  the  pre-service  teachers’  level  of  knowledge 
onthe  PISA-D? 

III.  LITERATURE  REVIEW 

Programme  for  International  Student  Assessment  for 
Development  (PISA-D)is  comparisons  of  15  years  old 
students’  knowledge  of  reading,  mathematics  and  scientific 
literacy  (OECD,  2001).  It  is  a  two-hour  test  that  students 
complete  with  pencil  and  paper  (Rinzin,  2019).  It  provides  a 
platform  for  policy  construction,  mediation,  and  diffusion  at 
the  national,  international,  and  even  global  level 
(Rizvi&Lingard,  2006).  PISA  measures  students’  skills  in 
applying  what  they  have  learned  in  school  to  real  life 
situations.  It  is  concerned  with  what  students  can  do  with 
what  they  had  learnt.  The  Director  General  of  the  Ministry  of 
Education  pointed  out  that  the  test  will  help  find  out  the 
educational  standard  of  the  country  through  comparison  with 
the  performance  of  studnets  in  other  PISA  member  countries. 
The  result  of  the  test  will  serve  as  evidence  for  policy  makers 
as  the  Ministry  of  Education  how  education  system  can  be 
improved.  Further,  the  result  of  the  test  will  also  be  used  as  “ 
appraisal  mechanism  on  improving  teacher’s  instructional 
and  classroom  management  practices”  (Rinzin,2019, 
para,20). 

According  to  Bhutan  Education  Blueprint  (2014),  Bhutanese 
Education  System  aspires  to  partake  in  international 
benchmarking  system  such  as  PISA  in  order  to  improve 
access,  equity,  system  efficiency  and  the  overall  quality  of 
education.  Recognizing  the  importance  of  participating  in 
international  benchmarking  systems  to  check  the  state  of 


education  in  the  country  against  the  international  standards 
and  acknowledging  its  need  at  the  earliest,  the  Ministry  of 
Education  endorsed  and  signed  the  Memorandum  of 
Understanding  (MoU)  for  PISA  participation  on  12th  January 
2017  during  the  18th  National  Education  Conference  in  the 
presence  of  His  Excellency  LyonchenTsheringTobgay,  the 
Prime  Minister  and  LyonpoNorbuWangchuk,  the  Minister 
for  Education. Therefore  2457  students  randomly  selected 
from  53  schools  across  the  country  have  been  tested  in 
reading,  mathematics  and  scientific  literacy  in  a  duration  of 
two  hours  test  conducted  in  November,  2017.  The  test  was 
followed  by  thirty  minutes  background  questionnaire.  The 
test  and  the  background  questionnaire  were  prepared  by 
OCED.  The  main  aim  of  participating  in  PISA-D  was  to 
mark  the  baseline  of  competencies  and  knowledge  of  the 
students  as  the  country  has  not  participated  in  an 
international  test  before. 

The  result  of  the  test  revealed  that  the  quality  of  education 
was  below  average  (Lamsang,  2018).  In  the  scale  of 
proficiency  level  from  1  to  6,  reading  and  mathematical 
literacy  was  found  to  be  below  level  1  and  scientific  literacy 
to  be  at  level  2.  For  example  in  reading  literacy,  Bhutan 
scored  45.34  %  while  one  of  the  top  performing  OECD 
member  countries  Ecuador’s  score  is  52.66%.  However,  It 
was  better  with  Mathematics  and  scientific  literacy  as  Bhutan 
score  came  to  second  position  with  38.84  percent 
(Rinzin,2019). While  comparing  the  standard  of  Bhutanese 
students  with  the  OECD  countries,  PISA  proficiency  level  is 
found  to  be  below  average  (Lamsang).  The  report  of  the  test 
further  called  for  “strengthening  competency  based  activities 
in  the  curriculum”  and  also  recommended  teacher  education 
colleges  (at  Samtse  and  Paro)  “to  incorporate  competency 
based  teaching  and  learning  modules  in  pre-service  training 
programme”  (Rinzin,  2019,  para  2). 

With  the  initiative  of  participating  in  international  test,  the 
schools  across  the  kingdom  that  caters  schooling  to  the  grade 
nine  students  were  mandated  to  institute  PISA-D  club  and 
conduct  tests  as  a  part  of  their  instructional  schedule.  In  some 
districts,  the  PISA-D  programmes  had  already  been  started  as 
a  part  of  the  school  curricula  and  every  one  hour  of  the 
Saturdays  are  devoted  to  the  PISA-D  club  so  as  to  coach  or 
conduct  classes  for  all  the  three  domains  of  reading,  science, 
and  mathematics  alternatively(  Wangdi,  Dahal.Tshomo, 
&Dorji,  2018).  However,  when  PISA  -D  ideas  are  put  into 
practice,  it  is  important  that  intent  or  the  aim  of  of  the 
participation  needs  to  be  clear.  As  per  the  main  reason  or 
purpose  of  PISA-D  is  to: 
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test  how  well  young  people  are  — prepared  for 
future  challenges,  whether  — they  can  analyze, 
reason  and  communicate  effectively!,  whether 
they  have  — the  capacity  to  continue  learning 
throughout  life?  And  to  what  extent  they  have 
acquired  some  of  the  knowledge  and  skills  essential 
for  full  participation  in  society.  (Sjpberg,  2014, 

p.120). 

IV.  METHODOLOGY 

The  study  adopted  a  mixed  method  of  qualitative  and 
quantitative  approach  to  explore  the  opinions  (awareness, 
perceptions,  attitudes,  interest  and  knowledge)  of  pre-service 
teachers  on  PISA-D.  The  26  survey  questionnaires  were 
prepared  and  administered  to  all  B.Ed.  fourth  year  and  PgDE 
students  of  Samtse  college  of  Education  to  assess  their 
awareness,  perception,  attitude,  interest  and  knowledge. 
From  23  survey  questionnaire,  16  were  Likert  scale  items 
and  seven  were  binary  items.  Four  students  (two  males  and 
two  females  coded  as  student  1,  student  2  etc.)  were 
interviewed  to  explore  their  perception  towards  PISA-D  and 
Bhutan  taking  part  in  it.  The  participation  was  voluntary. 

The  survey  questionnaires  were  analyzed  by  Add-in  Analysis 
Tool  Pak  and  Interview  data  were  analyzed  by  identifying 
themes. 


V.  RESULTS 

1.  Awareness 

From  the  survey  data  it  is  revealed  that  72.6%  of  pre-service 
teachers  at  Samtse  College  have  heard  about  the  PISA-D  test 
but  27.4%  of  them  were  not  aware  of  the  same.  With  regard 
to  the  knowledge  of  PISA  ranking  of  the  member  countries 
only  17.7%  of  pre-service  teacher  were  aware.  The  survey 
further  revealed  that  only  35  pre-service  teachers  were  aware 
of  two  preliminary  tests  that  were  conducted  in  the  country. 
However,  34  pre-service  teachers  were  aware  that  Bhutan 
will  be  joining  PISA-D  in  2024.  Despite  the  availability  of 
social  media  through  which  the  information  of  PISA-D  has 
been  communicated  yet  majority  of  the  pre-service  teachers 
(37.1%)  were  informed  about  it  by  tutors.  Student  1  said  that 
she  came  to  know  about  PISA-  D  only  when  her  teacher 
talked  about  it  in  the  class.  In  case  of  Student  2,  he  became 
aware  of  PISA-  D  from  his  parents  (who  are  teachers)  at 
home  when  he  overheard  their  talk  about  PISA-  D.  Only 
3.2%  of  them  were  informed  by  media  such  as  newspaper.  It 
shows  that  the  use  of  media  for  getting  information  is  poor. 
The  students  reliance  on  tutors  and  their  own  friends  for 
information  is  quite  significant  as  it  can  be  seen  in  figure  1 . 
Some  of  the  students  knew  about  PISA  -D. 


BBS  Newspaper  College  Tutors  Friends  Other 


Source  of  Information 


Fig.l.  Pre-service  teachers  preferred  source  of  information  about  PISA 
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2.  Perceptions  and  attitudes  of  pre -service  teachers  towards  PISA  -D 


Tablel.  Perceptions  and  Attitudes  of  pre-service  teacher  towards  PISA-D  in  percentage 


Strongly 

Agree 

Agree 

Not  sure 

Disagree 

Strongly 

Disagree 

PISA-D  gives  a  clear  picture  of  quality  of  education 
system  in  the  country 

15 

40 

44 

1 

1 

Bhutan  should  join  PISA 

27 

42 

31 

0 

0 

PISA  result  will  lead  to  change  in  curriculum 

21 

40 

38 

2 

0 

PISA  result  will  have  impact  on  teachers 

14 

48 

33 

5 

0 

PISA-D  results  will  affect  my  teaching  approaches 

8 

32 

51 

8 

1 

Teachers  should  have  enough  knowledge  of  PISA-D 

31 

47 

22 

0 

0 

I  am  interested  to  know  the  PISA-D  ranking  of  my 
country 

50 

34 

13 

2 

0 

PISA-D  test  will  raise  the  standard  of  education  in 
the  country 

18 

48 

35 

0 

0 

I  want  to  know  more  about  PISA-D  test 

60 

30 

10 

0 

0 

42%  of  the  pre-service  teachers  agreed  that  Bhutan  should 
join  PISA-D.  Most  pre-service  teachers  agreed  that  impact  of 
PISA-D  is  high  on  curriculum  (40%)  and  the  quality  of 
education  (40%)  in  the  country.  However  more  than  half  of 
them  (51%)  were  not  sure  whether  PISA-D  will  affect  their 
teaching  methods.  Majority  of  the  pre-service  teachers  (60%) 
want  to  know  more  about  PISA.  47%  of  them  agreed  that 
teacher  should  have  enough  knowledge  on  PISA-D 


50%  of  them  were  interested  to  know  the  PISA  ranking  of 
other  countries(see  Table  1). 

3.  Knowledge  of  pre-service  teachers  on  PISA-D  test 

On  average,  more  than  half  of  pre-service  teachers  (58.7%) 
were  not  sure  what  PISA-D  actually  is  and  15.3%  of  pre¬ 
service  teachers  have  low  level  of  knowledge  on  PISA-D 
test.  Only  9.1%  of  pre-service  have  little  knowledge  on  the 
PISA-D. 


Table  2.  Knowledge  of  Pre-service  teachers  on  PISA-D 


Strongly 

Agree 

Agree 

Not  sure 

Disagree 

Strongly 

Disagree 

PISA  D  is  international  assessment 
of  knowledge  and  skills  of  15  years-old  student 

7.2 

24.2 

61.3 

6.5 

0.8 

PISA-  D  will  focus  on  science, 
mathematics  and  reading  skills 

13.7 

26.6 

56.5 

2.4 

0.8 

PISA-D  is  not  just  a  test,  but  is  also  part  of  international 
curriculum  work  based 

on  aggregated  comparative  data  of  student  ability  in 
science  maths  and  reading  of  different  countries 

11.3 

37.1 

50.8 

0.8 

0 

PISA-D  test  focuses  on  15 -year-olds  to  check  whether  they 
can  apply  what  they  have  learned  in  school  in  real  life 
situations  or  not 

4 

25.8 

66.1 

4 

0 

Mean 

9.1 

28.4 

58.7 

13.7 

1.6 

Though  the  survey  revealed  very  low  level  knowledge  of  pre¬ 
service  teachers  having  proper  understanding  of  the  concept 
of  PISA-  D,  yet  the  interview  revealed  that  the  four 
interviewees  did  knew  its  importance  in  the  school  education. 


In  the  interview  Student  1  expressed  that  PISA-  D 
assessment  evaluates  students’  learning  achievement.  How 
much  of  the  learning  has  happened  in  the  class  could  be 
revealed  in  the  standardized  test  as  PISA-D.  He  related  that 
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the  teaching  should  be  guided  by  knowledge  of  PISA  D  so 
that  teachers  stress  importance  on  enhancing  the  competency 
students  rather  than  focusing  how  much  they  can  memrorise 
or  keep  concepts  in  their  mind.  Student  2  also  expressed  the 
importance  of  PISA  D  to  pre-service  teachers  since  it  is  to  be 
used  to  evaluate  the  overall  standard  of  Bhutanese  students 
with  the  students  of  other  countries.  Student  1  also  had  the 
same  view  that  preservice  teachers  should  have  knowledge  of 
PISA-D.  He  said,”  it  has  become  part  of  education  system  in 
the  recent  times  and  it  talks  of  international  standard 
assessments  which  is  mainly  to  do  with  competency  than 
content.”  He  also  said  that  “final  students  will  soon  become 
teachers  and  in  the  field  if  people  talk  of  PISA-D  then 
without  any  familiarity  done  in  the  college,  it  will  be  quite 
difficult  to  implement  PISA-D  aspect”.  Student  4  expressed 
that  if  pre-service  teachers  are  taught  about  PISA-D  before 
the  students  become  teachers,  it  will  help  them  to  contribute 
enhancingthe  quality  education  in  the  schools.  She  said,  “it 
is  very  important  for  the  trainees  to  know  about  PISA-  D 
because  through  PISA-D,  once  can  understand  the  level  the 
students  are  at  and  so  will  help  while  teaching.  Through 
PISA-  D  findings  one  can  get  information  that  students  who 
are  not  that  good  at  higher  order  thinking  will  be  good  at 
answering  lower  order  question.  Through  that  teachers  can 
try  to  get  the  students  analyse  and  think  and  understand,  not 
just  like  what  they  have  studied  but  what  they  have 
understood  and  can  apply.” 

According  to  Student  3,  “it  is  important  to  know  about  PISA- 
D  because  once  we  graduate  and  become  teachers,  we  will  be 
in  direct  contact  with  the  students.  If  we  have  right 
knowledge  and  information  on  PISA-D,  we  will  be  able  to 
guide  our  students  since  it  focus  on  students’  application 
skills.  We  can  help  students  improve  their  standards  and 
meet  the  level  of  international  benchmarks.” 

VI.  DISCUSSION 

Teachers  are  the  main  stakeholder  involved  in  development 
and  improvement  of  education  in  the  nation.  The 
improvement  and  development  of  the  education  can  be 
measured  by  PISA.  Therefore  teacher  should  be  well 
informed  of  PISA,  show  interest  in  the  result  and  integrate 
the  research  findings  in  educating  the  students.  However  the 
study  found  out  that  27.4%  of  pre-service  teachers  were  not 
aware  of  PISA  which  is  a  huge  number.  More  than  half  of 
pre-service  teacher  were  not  sure  what  the  PISA  really  is. 
When  focus  comes  to  the  source  of  information  about  the 
PISA,  tutors  played  important  role  in  term  of  knowledge 
acquisition.  Therefore  the  familiarization  of  pre-service 


teachers  to  PISA  in  the  college  is  crucial.  According  to 
Student  3,  “there  is  no  particular  module  which  talks 
specifically  on  PISA  D.  It  should  be  in  some  ways  put  in 
where  pre  service  teachers  are  made  familiar  on  PISA  D.” 
He  also  said  that  “  with  even  the  Bhutan  Council  for  School 
Examinations  and  Assessment  (BCSEA)  setting  certain 
portion  of  the  question  paper  based  on  competency  known  as 
Competency  Based  Question  (CBQ),  the  pre-service  teachers 
should  be  given  skills  of  designing  CBQ  questions”  Thus 
thePISA  report  also  calls  for  strengthening  and  enhancing 
competency  based  activities  and  assessment  in  curriculum 
and  recommends  teacher-training  colleges  to  incorporate 
competency-based  teaching  and  learning  modules  in  pre¬ 
service  training  programme  (BHSEC,  2019). 

As  for  the  impact  of  PISA,  most  pre-service  teachers  agreed 
that  impact  of  PISA-D  is  high  on  curriculum  (40%)  and  the 
quality  of  education  (40%)  in  the  country. 

VII.  FINDINGS  AND  RECOMMENDATIONS 

The  study  revealed  that  there  is  low  level  of  knowledge  and 
understanding  of  PISA-D  by  preservice  teachers  despite 
PISA-  D  being  talked  about  in  the  media.  The  survey 
revealed  that  it  was  mainly  through  the  initiatives  of  tutors 
that  the  pre-service  teachers  came  to  know  about  PISA-D. 
However,  the  few  who  did  had  some  knowledge  and 
information  knows  the  importance  of  PISA-D. 

Therefore,  it  is  recommended  that  pre-service  teachers  be 
made  aware  of  PISA-D  as  it  is  one  of  the  initiatives  taken  by 
the  Ministry  of  Education  to  put  effort  towardsenhancing 
competencies  of  students’  learning.  In  the  Assessment 
module  and  in  the  assessments  of  other  subject  specific 
modules  the  CBQ  model  of  designing  assessment  can  be 
integrated  and  implemented  so  that  what  pre -service  students 
learn  is  aligned  with  the  model  or  design  of  BCSEA 
questions.  It  has  also  been  reported  in  the  Kuensel,  the 
national  newspaper  that  ‘competency  based  activities  and 
assessments’  be  integrated  into  curriculum  in  teacher  training 
colleges. 
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